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FOR  PUBLIC  RELEASE 

Abstract 

Unbiased  Cramer-Rao  lower  bound  (CRB)  theory  can  be  used  to  calculate  lower  bounds  to  the  variances  of 
unbiased  estimates  of  a  set  of  parameters  given  only  the  probability  density  function  of  a  random  vector  conditioned 
on  the  true  parameter  values.  However,  when  the  estimated  parameter  values  are  required  to  satisfy  inequality 
constraints  such  as  positivity,  the  resulting  estimator  is  typically  biased.  To  calculate  CRBs  for  biased  estimates  of 
the  parameter  values,  an  expression  for  the  bias  gradient  matrix  must  also  be  known.  Unfortunately,  this  expression 
often  does  not  exist.  Because  expressions  for  biased  CRBs  are  preferable  to  sample  variance  calculations,  alternate 
methods  for  deriving  biased  CRB  expressions  associated  with  inequality  constraints  are  needed.  Here  we  present  an 
alternate  approach  that  is  based  upon  creating  the  probability  density  function  associated  with  a  given  biased 
estimate  of  these  parameters  using  the  available  knowledge  of  the  estimator  properties.  We  apply  this  approach  to 
the  calculation  of  biased  CRBs  for  estimators  that  use  a  positivity  constraint  with  and  without  a  support  constraint 
for  a  specific  measurement  model  and  discuss  the  benefits  and  limitations  of  this  approach. 

c 

1.  Introduction 

Wh<;n  ^eekir|g  t0  estimate  a  vector  of  unknown  parameters  from  a  set  of  measurements,  it  can  be  useful  to 
be  able  to  calculate  lower  bounds  to  the  variances  of  any  estimates  of  these  parameters.  One  potential  use  of  these 
bounds  is  to  determine,  a  priori,  whether  or  not  the  desired  estimation  accuracy  can  be  achieved  for  a  given 
measurement  model.  Another  use  is  to  employ  them  in  designing  an  experiment  for  which  it  is  known  that  the 
desired  accuracy  can  be  achieved.  A  third  use  for  these  types  of  bounds  is  to  check  the  accuracy  of  a  software 
implementation  of  an  estimation  algorithm  by  comparing  its  results  to  the  lower  bounds. 

One  well-known  theory,  called  the  unbiased  Cramer-Rao  lower  bound  (CRB)  theory,1  provides  algorithm- 
independent  lower  bounds  (called  unbiased  CRBs)  to  the  variances  of  any  unbiased  estimates  of  a  vector  of 
parameters  obtained  from  a  set  of  measurements.  These  bounds  can  be  calculated  using  only  the  probability  density 
function  (PDF)  of  the  measurements  that  is  a  function  of  the  true  parameter  values.  The  unbiased  CRBs  may  not  be 
achievable,  but  often  they  can  be  achieved  or  at  least  approached  closely.  However,  in  many  cases,  unbiased 
parameter  estimates  are  not  possible  (non-invertible  system  models)  or  desirable  (excessive  noise  amplification  in 
the  unbiased  estimates).  For  biased  parameter  estimates,  biased  CRBs  can  be  calculated.1  In  addition  to  the 
measurement  PDF,  the  bias  gradient  matrix  of  the  parameter  estimates  is  needed  to  calculate  biased  CRBs.  We  note 
that  an  alternate  approach  to  biased  CRB  theory  is  available  that  generates  biased  CRBs  (called  uniform  CRBs)  that 
depend  upon  only  the  norm  of  the  bias  gradient  and  not  its  direction.2 

Our  interest  in  CRB  theory  is  to  use  these  lower  bounds  to  quantify  how  the  various  types  of  knowledge 
included  in  parameter  estimation  algorithms  affect  the  qualify  of  the  parameter  estimates.  There  exists  a  large 
number  of  possible  types  of  knowledge  that  can  be  included  in  the  estimation  process  (c.f.  Ref.[31),  depending  upon 
the  type  of  input  signal  and  the  system  through  which  it  passes  prior  to  measurements  being  taken.  Two  types  of 
knowledge  that  have  been  used  extensively  in  parameter  estimation  algorithms  are  the  support  constraint  (restricting 
the  solution  set  to  parameters  that  are  only  non-zero  in  a  specified  region)  and  the  positivity  constraint  (also  known 
as  non-negativity;  i.e.,  restricting  the  solution  set  to  parameters  that  are  not  negative).  The  support  constraint  is  a 
speci  ic  example  of  a  more  general  class  of  constraints  known  as  equality  constraints,  where  functions  of  the  noise- 
free  parameters  are  known  to  be  zero.  Similarly,  the  positivity  constraint  is  a  specific  example  of  a  class  of 
than  zero^  k*0'”1  “  lnequallty  C0nstraints’  where  functions  of  the  noise-free  parameters  are  known  to  be  greater 

In  previously-published  seminal  work,4  a  detailed  CRB  theory  was  developed  that  quantified  the  ability  of 
equality  and  inequality  constraints  to  reduce  noise  in  parameter  estimates  for  estimators  with  a  given  bias  gradient 
matnx.  One  key  finding  in  this  work  is  that  imposing  equality  constraints  on  the  parameters  to  be  estimated  can 
produce  lower  CRBs  than  without  the  equality  constraints.  A  second  key  finding  is  that  inequality  constraints  on  the 
parameters  to  be  estimated  cannot  produce  lower  CRBs  as  long  as  the  bias  gradient  matrix  remains  unchanged  This 

con^int^ft8  ‘S  a'  first  PcrPiexmg.  because  it  is  well  known  to  signal  processing  practitioners  that  a  positivity 
constraint  often  produces  less  noisy  signal  reconstructions  than  possible  without  a  positivity  constraint.  The  answer 
to  this  conundrum  is  that  the  variance  reduction  brought  about  by  inequality  constraints  on  the  parameters  is 
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where  In  denotes  the  natural  logarithm  and  E[]  denotes  the  expected  value  of  the  quantity  in  the  brackets.  Here  we 
assume  that  the  FIM  is  invertible;  if  not,  either  a  subset  of  or  functions  of  the  parameters  may  be  estimable  in  an 
unbiased  way.  ’  As  can  be  seen  from  Eq.(l),  the  FIM  is  a  function  only  of  the  measurement  PDF  and  thus  so  are  the 
unbiased  CRBs. 

Biased  CRBs  can  be  calculated  in  terms  of  the  FIM  defined  in  Eq.(l)  using  the  following  expression 

cRBb,Me»  =  diag{[l  +  Veb(e)]F  '(eXl  +  Veb(e)]T },  (2) 

where  b(0)  is  the  bias  and  Veb(0)  is  the  bias  gradient  matrix  associated  with  a  given  estimator  that  is  a  function  of 
0,  and  I  is  the  identity  matrix  of  appropriate  dimension.  Obviously,  an  expression  for  Veb(o)  is  needed  to  carry  out 
the  calculations  in  Eq.(2)  in  addition  to  the  measurement  PDF. 

When  it  is  known  that  the  true  parameter  values  satisfy  a  set  of  equality  constraints,  the  FIM  can  be 
modified  to  include  this  additional  information  and  improve  the  accuracy  of  both  unbiased  and  biased  CRBs.  The 
general  theory  on  how  to  modify  the  FIM  to  include  equality  constraints  can  be  found  in  Ref.  [4],  For  the  special 
case  of  a  support  constraint,  the  constrained  FIM  is  obtained  by  zeroing  out  the  rows  and  columns  of  the 
unconstrained  FIM  corresponding  to  the  elements  of  0  outside  the  known  support.  Then  F'1  in  Eq.(2)  is  replaced  by 
its  pseudoinverse.  No  changes  need  to  be  made  to  Veb(o) . 

When  it  is  known  that  the  true  parameter  values  satisfy  a  set  of  inequality  constraints,  the  FIM  remains 
unchanged,  however,  the  estimator  bias  gradients  are  altered  when  the  inequality  constraints  are  active.  In  this  case, 
Veb(o)  must  be  modified  to  have  the  biased  CRBs  reflect  the  inequality  constraint.  Unlike  for  equality  constraints', 
no  general  theory  is  available  that  describes  how  to  modify  an  existing  V6b(0)  to  include  the  inequality  constraints. 
Finally,  when  both  equality  and  inequality  constraints  are  jointly  known  about  the  true  parameter  values,  the 
associated  CRBs  are  calculated  by  separately  modifying  the  unconstrained  FIM  to  reflect  the  equality  constraints 
and  the  bias  gradient  matrix  to  reflect  the  inequality  constraints. 

3.  Estimation  PDF  creation 

In  this  section,  we  describe  first  how  estimation  PDFs  can  be  used  to  obtain  CRB  expressions  that  are 
equivalent  to  the  standard  CRB  expressions  that  are  obtained  using  measurement  PDFs.  Second,  we  show  how  to 
create  estimation  PDFs  for  three  different  estimators:  (1)  one  that  employs  a  positivity  constraint,  (2)  one  that 
employs  a  support  constraint,  and  (3)  one  that  employs  both  a  positivity  and  a  support  constraint.  For  all  three  cases, 
we  assume  that  the  FIM  is  invertible.  Third,  we  describe  conceptually  how  to  extend  the  estimation  PDF  creation 
method  to  more  general  inequality  and  equality  constraints  as  well  as  to  non-invertible  FIMs.  Finally,  we  use  our 
method  to  create  estimation  PDFs  for  a  simple  measurement  model  and  the  three  different  estimators  described  in 
this  section  in  order  to  demonstrate  how  to  apply  the  estimation  PDF  creation  process.  In  Section  4,  these 
expressions  will  be  used  to  calculate  and  evaluate  CRBs. 

3.1  CRB  calculations  using  estimation  PDFs 

To  motivate  the  benefits  and  validity  of  creating  a  estimation  PDF  for  CRB  calculations,  consider  first  a 
standard  result  from  probability  theory.7  Let  y  be  a  random  vector  and  g(y)  be  an  arbitrary  vector  function  of  y.  The 
expected  value  of  g  can  be  calculated  in  one  of  two  equivalent  ways: 

E[g(y)]=  Jg(y)f(y;0)dy 

=  Jzf(z;0)dz  ^  ^ 

where  z-  g(y)  and  f(z;0)  is  the  PDF  of  z  that  is  a  function  of  0.  Because  z  is  the  independent  set  of  variables  in  z 
space,  no  knowledge  of  g(y)  is  necessary  for  calculating  the  desired  expected  value  when  using  the  second  integral 
expression  in  Eq.(3).  In  other  words,  f(z;0)  incorporates  all  the  information  contained  in  g(y).  When  a 
mathematical  expression  for  g(y)  is  available,  f(z;0)  can  be  calculated  from  f(y;0)  using  standard  PDF 
transformation  methods.7-8 

Phis  same  principle  can  be  used  to  create  a  biased  CRB  expression  that  is  equivalent  to  the  expression 
given  in  Eq.(2).  To  use  Eq.(2),  an  expression  for  the  bias  gradient  matrix  Veb(0)  must  be  available.  In  practice. 
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3.2.1  Positivity-constrained  estimation  PDF  creation 

Let  0p  be  a  positivity-constrained  estimator  of  0  with  PDF  fp(ep;o).  We  define  0p ,  element  by  element, 

as  follows:  we  set  the  j,h  element  of  0  ,0  equal  to  the  ith  element  of  ft  o  if  a  :c  „ 

p>  pj’  qumiumej  element  or  ,  0uj ,  if  0uj  is  non-negative,  otherwise 

we  set  0pj  equal  to  zero.  Of  course,  this  is  not  the  only  way  that  positivity  can  be  enforced.  Other  ways  include 
replacing  negative  numbers  with  their  absolute  values  or  adding  a  positive  constant  that  is  larger  than  the  absolute 
h  “„L°  h«PD?'^ma8n,tUdC  number.  The  uSe  of  one  of  these  alternate  ways  to  enforce 

n,If  p°/i't|V|ty “Constrained  e!,lma,or  and  lhus  d“igc  the  CRBs.  This  issue  is  not  explored  taller 
because  the  focus  of  this  paper  is  on  the  PDF-creation  process,  not  on  the  optimal  way  to  enforce  positivity.  In 

addition,  we  require  that  the  statistical  properties  of  0p  be  consistent  with  the  statistical  properties  of  0  as  much  as 

possible  while  satisfying  the  positivity  constraint.  The  latter  requirement  can  be  understood  by  realizing  that  a 
onstrained  estimate  of  a  set  of  parameters  is  chosen  to  be  as  consistent  as  possible  with  the  unconstrained  Estimate 

while  also  satisfying  the  known  constraints.  We  will  now  create  fp(ep;0)  based  upon  this  description  of  the 
estimator.  When  all  the  elements  of  0u  are  non-negative,  the  positivity  constraint  is  not  active  and  thus  fp(0p;0)  = 

f>u;0)  for  these  values.  However,  when  one  or  more  elements  of  0u  are  negative,  the  positivity  constraint  is 
active  and  this  equality  does  not  hold.  When  the  vector  0  is  N-dimensional,  there  are  2N-1  possible  combinations  of 
the  elements  of  0u  for  which  the  positivity  constraint  is  active.  As  a  result,  there  are  2N  terms  in  the  expression  for 
fp(op;e).  We  will  first  create  an  expression  for  the  term  in  fp(ep;o)  that  corresponds  to  the  case  where  only  the  first 
element  of  0u ,  0ul ,  is  negative.  Based  upon  this  result,  we  will  then  present  a  general  expression  that  can  be  used 
to  generate  any  term  in  fp(op;o)  except  for  the  case  where  all  the  elements  of  0u  are  negative.  This  last  case  can  be 
obtained  by  subtracting  the  integral  of  the  sum  of  all  the  other  terms  of  fp(ep;o)  from  one  since  PDFs  integrate  to 

one.  We  note  that  the  number  of  terms  in  fp(dp;o)  can  become  quite  large.  In  Section  5,  we  address  potential  ways 
to  deal  with  this  problem. 

The  PDF  creation  process  can  be  achieved  with  the  help  of  the  PDF  form  of  Bayes’  theorem.7  When  only 
the  first  element  ot  0p  has  been  set  equal  to  zero  because  of  the  positivity  constraint,  Bayes’  theorem  can  be  used  to 

express  the  constrained  joint  PDF  fp(opl  =  O,0pJ,...,0pN;0)  in  terms  of  its  associated  constrained  conditional  PDF 
fp(0p:.-0PN|0p,  =0;©)  and  constrained  marginal  PDF  )•••  )fp(0pl  =0,ep2,...,epN;o)dep2-depN  as  follows:7 

-oo  -» 

fp(®pi  =O’0P2 . 0pN;0)=fp(eP2,...,0pN|0pl  =0;©)j •••  ]fp(epl  =o,0p2,...,epN;o)d0p2-d0p 


‘'pN  • 


(7) 


Based  upon  Eq.(7),  we  can  create  fp(op|  =  O,0p2,...,0pN;0)  by  creating  separately  fp (e„2 0pN I GpI  =O;0)  and 
J  Jfp(®pi  =  O>0p2,...,epN;o)d0p2--depN  and  then  multiplying  the  two  expressions  together. 

-oo  -oo  0 

from  a  closer  look  at  Eq.(7),  we  can  see  that  the  constrained  conditional  PDF  carries  the  statistical 
information  about  Op  when  Opl  =  0  while  the  constrained  marginal  PDF  is  the  integral  over  all  the  values  of  0p  for 

which  0pl  =0.  We  desire  to  have  the  constrained  joint  PDF  be  as  consistent  with  the  unconstrained  joint  PDF  as 

nroSnebrlirh,fetShatlS<Ving  ** •‘T™  constraints'  Because  the  unconstrained  conditional  PDF  describes  the  statistical 
property  of  the  unconstrained  estimate  of  0,  we  satisfy  the  unconstrained  PDF  consistency  desire  by  setting 

fp\0p2.-.0pN|0pl  =O;0j  equal  to  the  unconstrained  conditional  PDF  fu ((9u2 ,...,0uN 1 0uI  =0;e).  The  difference 

between  the  constrained  and  unconstrained  joint  PDFs  is  thus  carried  in  the  marginal  PDFs.  We  obtain  the 
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ip(epl  o,...,epM  -o,ep(M+1) . epN;e)  =  fu(eu(M+o,...,0uN|eul  =  o,...,0uM  =  o;@) 

00  00  . 

J-  =°v.,)0UM  =0,eu(Mtl) . e^de^-de. 


,-oo  “-oo 
0  0  oo  oo 


u  «  w  w  /  . 

H  /'••|fu(0ul,..,0uN;0)d0u(M+1)-d0u 

-oo  -ao  |_-oo  — ao 

x8(0pi)-"s(ePM)u+(ep(M+1))-u+(0pN) 


•  do 


deu,-d0uM 


Because  M  is  arbitrary  and  the  elements  of  0u  can  be  arbitrarily  reordered,  Eq.(l  1)  can  be  used  to  calculate  all  the 
terms  that  are  part  of  fp(ep;o)  except  for  the  cases  of  no  negative  elements  and  all  negative  elements.  The  term 


corresponding  to  no  negative  elements  in  0u ,  fp(epl,...,epN;o),  is  given  by 

fp(0pi>--,0pN;0)= fu  ..,0uN;e)u+(0pl)--'U+(0pN), 
and  the  term  corresponding  to  all  negative  elements  0u ,  fp((§pl  =  O,...,0pN  =  0;©),  is  given  by 

fp(0pl  =  O,...,0pN  =  0; @)=  jl  -  f  )[sumof  alltheothertermsinfp(0p;0)]d0pl-  d0pN  j s(epl ) --5(epN ). 
Equations  (1 1>(13)  completely  define  fp(ep;o). 


(12) 


(13) 


Unlike  for  the  positivity  constraint,  the 


3,2.2  Support-constrained  estimation  PDF  creation 

Let  0s  be  a  support-constrained  estimator  of  0  with  PDFf  fe  e) 

s\  S,w/‘  -•*■“*»  pvainviiy  vuuaudllll,  me 

support  constraint  is  independent  of  the  measurement  values.  As  a  result,  for  a  given  support  region,  f  fe  -0) 
consists  of  only  one  term  that  will  now  be  created.  Let  there  be  M  elements  outside  of  the  support  region,  where  0  < 
-  N.  We  require  that  M  be  strictly  less  than  N;  otherwise,  there  are  no  elements  in  the  support  region  and  thus 
no  parameters  to  estimate.  As  before,  without  loss  of  generality  we  assume  that  the  first  M  elements  of  0  are  the 
constrained  elements.  Following  the  same  approach  to  create  fs(0s;0) 

decompose  f,(e,;e)s  f.fe,,  =O,...,0iM  =O,0l(Mtl) . 0iN;0)  into 

M®s<Mn)>"->0sN  |®si  =O»-  -,0sM  =  0;©)  and  marginal  PDF  f—  ff  fe  ,  =0  0  =0  0  ft  -ft) 

Mathematically,  we  have 


as  we  used  to  create  fp(op;0),  we  first 
a  product  of  its  conditional  PDF 


d0S(M+i)"'d0sN 


fs(e5l  o,...,esM  -o,es(M+1))...,esN;e)=fs(es(Mt|))...)esNj0!i  =o,...,0sM  =o;o) 

X  00  . 

X  J  J^sfesl  =0>— ’®sM  =  0’0S(M+l)v-,0sN;0)d0s(M4l)-  d0s] 


(14) 


For  the  same  reasons  as  given  in  Subsection  3.2.1,  we  set  the  support-constrained  conditional  PDF  equal  to  the 
unconstrained  conditional  PDF  fu (6u(M+1),...,0uN  | G . 0uM  =O;0),  where 
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given  by  Eq.(17),  the  suppon-only  case.  When  the  support  region  contains  all  the  elements  of  0u ,  K  =  0  and 
fps(®pS>®)  's  given  by  Eqs.(  1 1 )-( 13)  depending  upon  the  actual  values  of  the  elements  of  0 

3.3  Estimation  PDF  extensions 

The  estimation  PDF  creation  method  described  in  Subsection  3.2  was  developed  for  estimators  using  every 
combination  of  the  positivity  and  support  constraints.  The  essence  of  the  positivity  and  support  constraints  is  that 
estimated  parameter  values  are  constrained  to  be  zero  if  they  are  either  non-zero  (support)  or  less  than  zero 
(positivity).  However,  the  positivity  constraint  can  be  generalized  to  the  situation  where  each  parameter  value  is 
known  to  be  either  bounded  below,  bounded  above,  or  bounded  both  below  and  above  by  constants  that  can  vary  for 
each  parameter.  Similarly  the  support  constraint  can  be  generalized  to  the  situation  where  some  of  the  true 
parameter  values  are  already  known  exactly.  The  necessary  modifications  are  to  replace  the  zero  values  in  the 

|PDf  ex^essions  w,tb  the  known  equality  values  or  the  inequality  boundary  values  and  to  replace  the 
integration  limits  in  the  marginal  PDF  expressions  to  be  consistent  with  the  known  inequality  boundaiy  values. 

Equality  or  inequality  constraints  that  consist  of  combinations  of  true  parameter  values  cannot  be 
accommodated  using  this  estimation  PDF  creation  method,  though.  To  demonstrate  why  this  is  so,  consider  the  case 
where  the  sum  of  the  true  parameter  values  is  known  to  be  less  than  or  equal  to  a  known  value.  There  are  an 
uncountably  infinite  number  of  combinations  of  estimated  parameter  values  that  satisfy  this  constraint.  Because  the 
JJ®  h°?  d£STbtd  m  Subsectlon  3  2  creates  a  separate  expression  for  each  possible  combination  of  parameter  values 
that  satisfy  the  known  constraint,  this  inequality  constraint  obviously  cannot  be  accounted  for.  For  this  reason  the 
estimation  PDF  creation  method  can  be  used  only  when  constraints  on  individual  parameter  values  are  known. 

t'  ma  ui  tHe  efftimatlon  PDF  creation  method  can  be  extended  to  non-invertible  FIMs  by  recasting  the 
estimation  problem  to  be  invertible  and  carrying  out  the  method  on  this  recast  problem.  For  example  consider  the 
case  where  the  parameters  to  be  estimated  correspond  to  a  discretized  time  signal  and  the  measurements  are  a  low- 
pass-f,  tered  and  noisy  version  of  the  discretized  time  signal.  The  estimation  problem  can  be  recS  by  setting X 

the  H  ^  t0  3  low-Pass'flltered  version  of  the  original  true  set  of  parameters.  However, 

he  known  equality  and  inequality  constraints  on  the  true  parameter  values  must  be  able  to  be  transformed  to  the 
low-pass-filtered  parameter  set  for  this  approach  to  be  successful.  For  example,  a  positivity  constant  on  the 
original  set  of  parameters  is  still  valid  for  a  low-pass-filtered  set  of  parameters  if  the  low-pass  filter  is  chosen  to  be 
non-neganve.  A  support  constraint,  on  the  other  hand,  will  not  be  strictly  valid  for  a  low-pass-filtered  version  of  a 

nass  rraeHe'rS  !  ^  3  broadened  SUPP0rt  ieSio"  ca"  ^tam  virtually  all  of  the  low- 

pass  filtered  parameter  energy  and  is  often  applied  in  this  situation. 

3.4  Estimation  PDF  creation  example 

In  this  subsection  we  apply  the  estimation  PDF  creation  method  described  in  Subsection  3  2  to  a  specific 
measurement  model  and  three  different  estimators  in  order  to  demonstrate  how  to  carry  out  the  creation  process 
First,  the  measurement  model  will  be  described.  This  model  is  chosen  to  be  simple  irTorder  to 
described  in  the  previous  subsections  straightforward  to  illustrate.  Let  the  measurement  vector  y,  the  parameter 
vector  0,  and  the  noise  vector  t)  be  two-dimensional  and  be  related  by  the  following  equation 


y=e+n- 

where  the  noise  PDF  fN  (q)  is  zero  mean  and  Gaussian  with 


(19) 


a  covariance  matrix  given  by 


C  = 


_2  i  —2 

ct,  +  o2 
-.2  2 


(20) 


From  Eq.(  9),  ,t  can  be  seen  that  measurement  PDF  fY(y;e)  is  equal  to  fN(i,-y-0),  where  the  notation 
Mil  y  0)  denotes  the  PDF  fN(tj)  with  n  replaced  with  y-0.  The  relative  values  of  of  and  of  in  Eq  (20) 
define  the  measurement  statistical  correlation  properties.  When  they  are  equal,  the  two  components  of  the 
measurement  vector  y  are  uncorrelated.  When  of  =  0  ,  the  two  measurement  components  are  perfectly  correlated, 
and  when  of  -  0 ,  the  two  measurement  components  are  perfectly  anti-correlated.  We  note  that  this  definition  of  C 
is  not  conventional.  The  conventional  way  to  define  the  covariance  matrix  of  two  correlated  random  variables,  each 
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Jfv(yi,y2  =0;e)dy, 


1--2- 


]fv(y.  =0P5l,y2=o;e)d0psl 


Jfy(yi.y2  =O;0)dy, 


(23) 


Equations  (21)  -  (23)  can  be  used  to  calculate  the  FIMs  for  the  three  estimators  using  Eq.(l).  The  support- 
constrained  estimator  is  an  unbiased  estimator  of  6,  so  the  support-constrained  CRBs  are  just  the  diagonal  elements 
of  the  inverse  ot  the  support-constrained  F1M.  However,  both  the  positivity-constrained  and  positivity-  and  support- 
constrained  estimators  are  biased  estimators  of  0,  so  their  bias  gradient  matrices  must  be  calculated  using  Eq.(5)  and 
the  corresponding  constrained  PDFs  in  order  to  calculate  the  biased  CRBs  using  Eq.(6). 

4.  Biased  CRBs  for  positivity  and  support 

The  CRBs  corresponding  to  the  measurement  model  and  the  three  estimators  described  in  Subsection  3  3 
(positivity-constrained,  support-constrained,  and  both  positivity-  and  support-constrained)  are  analyzed  in  this 
section  as  functions  of  the  single-element  signal-to-noise  ratio  (SNR)  and  the  degree  of  correlation  as  determined  by 
the  ratio  a,  /a2 .  The  SNR  of  an  element  is  defined  as  the  noise-free  parameter  value  divided  by  the  square  root  of 

the  noise  variance.  We  compare  the  CRBs  to  sample  variances  obtained  using  computer  simulation  results  and 
show  that  the  CRBs  are  either  achieved  or  closely  approached. 

The  computer  simulation  results  presented  in  this  section  use  Eq.(19)  as  the  system  model  with  the  zero- 
mean  additive  noise  term  having  unit  variance  (i.e.,  o^+CTj  =1)  and  varying  values  of  of  and  a2  that  satisfy 

°|2  +  °2  =  1  •  The  values  of  0  were  varied  in  order  to  vary  the  single-element  SNR  values.  A  cost-function-based 
estimation  scheme  was  used  to  estimate  0  from  y  based  on  the  following  cost  function: 

j(a)  =  (y-a2)TC-'(y-a2),  (24) 

where  C  is  given  by  Eq.(20),  0  =  a‘ ,  and  the  minimization  was  performed  over  a.  This  parameterization  of  0 
enforced  the  positivity  constraint  while  still  permitting  an  unconstrained  (conjugate  gradient)  minimization  routine 
to  be  used.  The  unconstrained  estimator  corresponding  to  Eq.(24)  is  just  the  maximum-likelihood  estimator 
corresponding  to  the  measurement  PDF.  When  both  positivity  and  support  were  used  as  constraints,  one  element  of 
0  was  set  to  zero  and  the  minimization  was  carried  out  over  the  element  of  a  that  corresponded  to  the  non-zero 
element  of  0.  When  only  support  was  used  as  a  constraint,  the  a2  term  in  Eq.(13)  was  replaced  with  a  and  the 
minimization  was  carried  out  as  before.  Each  sample  variance  value  was  calculated  using  10,000  estimates  of  0 
generated  from  10,000  statistically-independent  measurements  for  a  given  set  of  values  for  o2 ,  ct2  ,  and  0. 

I  he  results  for  the  first  set  of  cases  to  be  presented  were  generated  for  0  =  [0.1,  0]T,  [1,  0]T,  and  [5  0]T 
while  varying  a2  /a2  from  0.01  to  100.  Because  the  noise  variance  in  each  element  is  one  regardless  of  the  value  of 
ct,  /ct2  ,  the  values  for  0  are  also  the  single-element  SNRs.  A  positivity  constraint  was  enforced  but  not  a  support 
constraint.  These  choices  of  values  and  constraints  correspond  to  estimating  parameter  values  both  inside  and 
outside  a  support  region  with  the  use  of  only  a  positivity  constraint.  The  resulting  biased  CRBs  and  sample 

variances  for  0,  (i.e.,  for  values  inside  the  support  region)  are  plotted  in  Figure  1,  where  the  CRBs  would  be  unity 
without  the  positivity  constraint.  Notice  the  good  agreement  between  the  sample  variances  and  the  biased  CRBs.  In 
addition,  when  ct2  /  a2  =1,  the  noise  values  in  the  two  measurement  elements  are  uncorrelated,  so  the  reductions  in 

the  biased  CRBs  and  sample  variances  corresponding  to  0,  for  this  value  of  ct2  /ct2  come  entirely  from  zeroing  out 
the  negative  values  in  y,.  Thus,  when  the  SNR  of  y,  is  five,  the  probability  of  obtaining  a  negative  value  for  y,  is 
quite  small,  so  the  biased  CRB  for  0,  in  this  case  is  essentially  equal  to  the  measurement  variance.  As  the  single- 
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variety  of  measurement  SNRs  and  noise  covariances.  Sample  variances  were  generated  for  the  same  set  of 
measurement  SNRs  and  noise  covariances  and  shown  to  match  closely  the  theoretical  CRB  values. 

An  important  outcome  of  this  work  is  the  explicit  recognition  of  the  role  that  noise  correlations  play  in 
determining  the  effectiveness  of  additional  knowledge  such  as  positivity  or  support  to  reduce  noise  in  the  estimated 
parameters.  This  result  provides  insight  into  how  positivity  and  support  will  function  to  reduce  the  noise  levels  in 
parameter  estimates  where  deconvolution  is  part  of  the  parameter-estimation  algorithm.  Positivity  and  support  are 
not  effective  in  reducing  noise  levels  in  estimated  parameters  when  the  noises  are  spatially  delta  correlated,  which 
includes  many  standard  noise  sources  such  as  amplifier  and  Poisson  noise.  However,  the  deconvolution  operation 
imposes  correlations  on  delta-correlated  noises,  leading  to  the  possibility  of  noise  reduction  in  this  case.  For  the 
case  of  a  support  constraint,  it  has  been  shown  that  meaningful  noise  reduction  is  possible  when  the  measurement 
model  includes  non-negligible  blurring.9 

Future  work  includes  extending  the  results  in  this  paper  to  delta-correlated  measurement  noises  and 
deconvolution.  Because  the  method  presented  in  this  paper  is  a  function  of  the  measurement  noise  values,  this 
extension  is  straightforward  in  concept  but  may  result  in  calculations  that  are  intractable.  Therefore,  future  work 
also  includes  developing  variations  to  the  method  presented  herein  that  are  computationally  friendly.’  One  way  to 
significantly  reduce  the  computational  burden  is  to  calculate  the  positivity-constrained  PDF  terms  only  for  elements 
whose  SNRs  are  less  than  three.  If  the  set  of  parameters  to  be  estimated  has  finite  support,  this  information  should 
always  be  included  because  the  support-constrained  PDF  terms  are  easy  to  calculate. 
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